Experimental Models for Validating Technology

E ffective software can mean software that is low cost, reliable, rapidly developed, safe, or has some other relevant attribute. To determine whether a particular technique is effective, we need a way to measure it. Merely saying that a technique is effective conveys no real information. Instead, we need measurements applied to each software attribute so we can say one technique is more or less effective than another. For some techniques, mapping from an effective attribute to a measurement scale is fairly straightforward. If “effective” means “low cost,” then cost of development would be such a measurement. For other attributes—such as reliability, safety, and security— measurements are not so easily taken. Measurements like the number of failures per day, errors found during development, or MTBF (mean time between failures) indicate reliability in hardware domains. For software, however, a count of the number of errors found during testing does not by itself indicate whether there are errors remaining to be found. Only refined experimentation can help determine that. Experimentation is a crucial part of attribute evaluation and can help determine whether methods used in accordance with some theory during product development will result in software being as effective as necCo m pu tin g Pr ac tic es